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Abstract 

Despite the emerging evidence base on response to intervention, there is limited research regarding how to effectively 
use progress-monitoring data to adjust instruction for students in Tier 2 intervention. In this study, we analyzed extant 
data from a series of randomized experimental studies of a kindergarten supplemental reading intervention to determine 
whether linking performance on formative assessments to curriculum progression improved kindergarten reading 
outcomes over standard implementation. We were interested in whether specific progression adjustments would enhance 
the effects of supplemental reading intervention. Growth-mixture modeling using data from kindergarteners (n = 136) 
whose intervention progression (e.g. repeat lessons, skip lessons) was adjusted every 4 weeks based on mastery data 
identified four latent classes characterized by unique profiles of curriculum progression adjustments. Multilevel analyses 
comparing the performance of students in the four classes with that of propensity matched groups whose intervention was 
not adjusted (n - 101) indicated positive effects of curriculum progression for (a) students whose formative assessment 
performance exceeded 90% and received early and sustained lesson acceleration and (b) students who initially performed 
below 70% on assessments and who repeated early lessons and progressed to conventional implementation. Effects of 
curriculum adjustments for the two smallest groups were less clear. 
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Response to intervention (RTI) in reading is an instructional 
framework that enables schools to provide the level and 
type of early intervention necessary to meet the needs of 
students with early academic risk. Grounded in prevention 
science and the evidentiary knowledge base in early read¬ 
ing, RTI in the primary grades provides a critical opportu¬ 
nity for early intervention, compared to later intervention, 
to differentially accelerate reading growth for children with 
early reading risk (Simmons et al., 2007; Vellutino, Scanlon, 
Small, & Fanuele, 2006; Vellutino, Scanlon, & Tanzman, 
1998). Since RTI was authorized in 2004 (Individuals with 
Disabilities Education Improvement Act [IDEIA], 2004), 
schools have increasingly embraced RTI practices. 
Specifically, a recent report on implementation of IDEIA 
indicated that more than 70% of elementary schools from a 
nationally representative sample use a version of RTI in 
reading/language arts (Bradley et ah, 2011). Typical fea¬ 
tures of RTI include (a) providing comprehensive high- 
quality classroom instruction, (b) screening all students to 


identify those who are at risk for future reading difficulties, 
(c) implementing evidence-based supplemental interven¬ 
tion for students who need additional support, and (d) inten¬ 
sifying the level of support for students who do not respond 
to supplemental intervention based on progress monitoring 
data (Bradley, Danielson, & Doolittle, 2005; National 
Center on Response to Intervention, 2010). 

In their review of RTI as implemented over the past 
decade, Fuchs and Vaughn (2012) concluded that although 
some RTI components have been incorporated into schools’ 
practice (e.g. screening), other essential features, including 
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progress monitoring, have failed to become widely adopted. 
In addition, these authors identified pressing issues that 
need to be addressed including research examining how to 
use progress monitoring to optimally inform instructional 
decisions. 

The need for research to complement the existing RTI 
knowledge base was also highlighted in the RTI practice 
guide by Gersten et al. (2009). In this guide, an expert panel 
concluded that there is a strong research base to support 
providing “intensive, systematic [reading] instruction ... in 
small groups who score below the benchmark score on uni¬ 
versal screening” (p. 6). The efficacy of Tier 2 intervention 
notwithstanding, a significant number of children do not 
demonstrate adequate response and require Tier 3 interven¬ 
tion. Of relevance to this study was how to intensify the 
effects of Tier 2 intervention to reduce the number of stu¬ 
dents who may require Tier 3 intervention. 

The purpose of the current study was to extend our 
understanding of the effects of using progress-monitoring 
mastery data to adjust the rate at which students pro¬ 
gressed through a Tier 2 intervention program. Specifically, 
we conducted secondary analyses with extant data from a 
series of randomized controlled trials to discern whether 
there was an overall effect of adjusting curriculum pro¬ 
gression based on student performance and to identify 
which unique student performance/curriculum progres¬ 
sion adjustments were most effective. In this study, we 
were interested in determining whether different student 
performance/curriculum progression profiles (e.g., accel¬ 
erated pacing, repeated lessons) enhanced the effects of a 
Tier 2 kindergarten reading intervention implemented 
conventionally (i.e., without adjustments). We hypothe¬ 
sized that examining the effects of specific adjustments 
would enable more effective engineering of interventions 
for students who do not respond adequately to standard 
implementation. 

Conceptual and Empirical Framework for Using 
Data to Adjust Reading Interventions 

Conceptually, using student perfonnance data to determine 
the pedagogical, instructional, and organizational variables 
that need to be manipulated is central to RTI (Lembke, 
McMaster, & Stecker, 2010; Stecker, Fuchs, & Fuchs, 
2008). Empirically, however, few studies have examined 
the effects of data-adjusted interventions. The RTI practice 
guide recommends using curriculum-embedded mastery 
checks to monitor student progress (Gersten et al., 2009) 
because the experimental research reviewed used these 
types of assessments to monitor progress. 

Curriculum-embedded measures (CEMs) are different 
from commonly used progress-monitoring materials that fall 
under the rubric of curriculum-based measurement (CBM). 
Unlike CBM probes, which assess student growth on a cur¬ 
riculum-independent general outcome (e.g., oral reading 


fluency), CEMs typically assess content that has been recently 
taught and measure mastery of specific skills. Thus, grounded 
in the theory of mastery learning (Bloom, 1971), CEMs are 
formative assessments that provide information to guide 
instructional modifications for individual students. CEMs are 
designed to complement general-outcome curriculum-based 
assessments by providing instructors infomiation to make 
adjustments based on an individual student’s performance. 

In a study of first-grade students, Denton et al. (2010) 
investigated the effects of a supplemental intervention, 
Responsive Reading Instruction (RRI; Denton & Hocker, 
2006). In the RRI condition, interventionists assessed stu¬ 
dents weekly to determine curriculum mastery and identify 
instructional needs. Based on mastery data, intervention¬ 
ists then selected instructional activities and curriculum 
adjustments for students, including progression through 
the curriculum, from a menu of options. Findings indicated 
that students in the RRI outperformed students in a control 
group. Of students in the business as-usual condition, 43% 
received a different intervention and 57% received no addi¬ 
tional intervention. Although this study provides evidence 
supporting RRI’s overall approach, it is difficult to identify 
which activities and adjustments contributed to improved 
student outcomes because RRI was implemented as a pack¬ 
age and compared to multiple variations of business as 
usual. 

In a study investigating the effects of individualized 
reading intervention for middle school students, Vaughn 
et al. (2011) used student performance on biweekly teacher- 
developed measures to determine skill mastery and guide 
instruction. They also used standardized monthly progress¬ 
monitoring measures to adjust intervention. Details on mas¬ 
tery levels for adjusting instruction were not discussed. 
Findings indicated no statistical or practical differences 
between the performance of students whose instruction was 
individualized based on performance data and that of stu¬ 
dents who received a standardized intervention. Students in 
both conditions outperformed students who received no 
additional intervention. 

To date, the converging body of research supporting 
tiered interventions is based on standard protocol reading 
interventions (Coyne, Kame’enui, Simmons, & Ham, 2004; 
Vadasy, Sanders, Peyton, & Jenkins, 2002; Vaughn et al., 
2011), in which students typically receive the same instruc¬ 
tional content at the same pace and with the same emphasis 
over a designated period. In their review of Tier 2 research, 
Fuchs and Vaughn (2012) concluded that the existing evi¬ 
dence supports standardized over problem-solving or indi¬ 
vidualized intervention approaches in reading. Advantages 
cited to support standardized approaches included better 
ability to document what students have been taught, more 
efficient use of resources, and increased opportunities to 
document implementation fidelity. 

Problem-solving or individualized interventions that use 
student data to dynamically adjust instruction inherently add 
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complexity to scientific inquiry because of the many moving 
parts involved. Prior studies (Denton et al, 2010; Vaughn 
et al, 2011) that have compared standardized to individual¬ 
ized approaches used intervention packages that differed 
across multiple dimensions (e.g., content of instruction, 
instructional emphasis, opportunity to learn). These studies 
are pragmatically important given the relatively few com¬ 
parative experimental studies that have been conducted. We 
propose, however, that there is middle ground between stan¬ 
dard and individualized intervention protocols consisting of 
using a consistent approach to adjust a student’s progression 
in an otherwise standard intervention. To understand the 
effects of adjusting intervention based on student perfor¬ 
mance, we sought to hold the intervention constant and 
manipulate a critical feature: progression in the curriculum 
based on student performance on CEMs. 

Previous Studies and Current Research 
Focus 

The study used extant data from a series of experimental 
studies to (a) identify whether unique performance/progres¬ 
sion profiles emerged for different groups of students when 
interventionists adjusted student progression through a Tier 
2 intervention based on student mastery data from forma¬ 
tive assessments and (b) examine the effects of different 
performance/progression profiles (e.g., accelerating lessons 
for students who score 90% or greater, repeating lessons for 
students who score less than 70%) on kindergarten reading 
outcomes. Data were drawn from four randomized trials 
that examined the effects of the Early Reading Intervention 
(ERI; Pearson/Scott Foresman, 2004), a Tier 2 kindergarten 
supplemental beginning reading intervention. Holding the 
intervention constant across studies enabled us to examine 
the effects of data-based adjustments. 

In the first two studies (Coyne et al., 2013; Simmons 
et al., 2011), researchers compared a conventional imple¬ 
mentation of ERI to school-designed Tier 2 interventions. 
In the first study, ERI groups outperformed school-designed 
intervention groups on a majority of measures. In the sec¬ 
ond study conducted in different schools, no statistically 
significant differences were found between ERI and school- 
designed intervention. Follow-up analyses indicated that 
ERI groups across both studies made similar gains; how¬ 
ever, in the second study, groups receiving strong business- 
as-usual interventions experienced similar outcomes. 

We hypothesized that a conventional implementation, 
where students progressed through the intervention at the 
same pace, may have impeded some students’ progress and 
that some students could have benefitted from repeated 
lessons and others from an accelerated pace. Therefore, in 
two subsequent studies, we examined whether adjusting 
intervention progression based on student performance 
would intensify ERI effects (Coyne et al., in press; Little 


et al, 2012). In the Coyne et al. (in press) study, we compared 
ERI that was implemented conventionally to ERI that was 
systematically adjusted over the year based on student per¬ 
formance. Between-group differences on a range of early 
reading measures including letter names, letter sounds, word 
identification, and oral reading fluency were statistically sig¬ 
nificant, with effect sizes ranging from 0.29 to 0.76 favoring 
ERI with ongoing adjustments. In the Little et al. (2012) 
study, we compared an adjusted ERI implementation version 
in a district to an organized RTI implementation. Results 
showed no statistically significant group differences on any 
outcome measures; however, effect sizes, ranging from 0.35 
to 0.59, revealed substantively important differences 
(Valentine & Cooper, 2003) favoring ERI with ongoing 
adjustments on multiple measures. 

Although these findings provided preliminary evidence 
that adjusting progression through the ERI program benefit¬ 
ted kindergarten students at early reading risk, these initial 
between-group analyses were unable to identify whether 
specific performance/progression adjustments were more 
effective than conventional implementation. That is, were 
overall differences attributable to specific groups of students 
whose progression was adjusted, such as students who did 
not demonstrate mastery on formative assessments and 
repeated targeted lessons? In the present study, we used data 
from students who received either ERI implemented con¬ 
ventionally or ERI implemented with ongoing adjustments 
from the four previously conducted experimental trials to 
better understand whether using formative assessment pro¬ 
files to adapt curriculum-progression would differentially 
improve end-of-kindergarten achievement. We defined “cur¬ 
riculum progression” as (a) accelerations of specific lessons, 
(b) repetitions of targeted lessons, or (c) conventional imple¬ 
mentation with targeted skill review. Decision-rule criteria 
for accelerating, maintaining, or repeating lessons are 
described later in this article. Through secondary analyses of 
data from four intervention cohorts, our research addressed 
two primary questions: 

1. What student performance/curriculum progression 
profiles (latent classes) were identified from differ¬ 
ent groups of students when ERI formative assess¬ 
ment data were used to accelerate, maintain, or 
repeat lessons? 

2. Which student performance/curriculum progression 
profiles enhanced end-of-kindergarten outcomes 
over the conventional ERI implementation? 

Method 

Participants 

Students included 237 kindergarteners who participated in 
one of four randomized control trials that examined the 
effects of variations of the ERI (Coyne et al., 2013; Coyne 



258 


Journal of Learning Disabilities 48(3) 


et al., in press; Little et al., 2012; Simmons et al., 2011). 
Students were divided into two groups: students who 
received ERI with ongoing adjustments (ER1-A) and stu¬ 
dents who received ERI implemented conventionally 
(ERI-matched peers). Data from students (n = 136) who 
received ER1-A (Studies 3 and 4) were used to identify pro¬ 
files of curriculum progression. Data from students (n = 
101) who received conventional ERI (Studies 1, 2, and 3) 
were identified as propensity matched peers (the details of 
the propensity score matching procedure are described in 
the analysis section). 

Participants were from three states (CT, FL, and TX) and 
26 public schools. Percentages of students at participating 
schools receiving free and reduced-cost lunch ranged from 
72% to 82% in Texas, from 3% to 81% in Connecticut, and 
from 31% to 92% in Florida. Enrollments ranged from 278 
to 985 students in the participating Texas schools, from 266 
to 749 in the Connecticut schools, and from 401 to 832 in 
the Florida schools. 

Across all four studies, researchers consulted with school 
personnel to identify children who (a) were considered in 
need of supplemental reading instruction based on school- 
administered reading assessments, (b) were at least 5 years 
of age, and (c) received reading instruction in English. 
Across the four cohorts, a two-step process was used to 
select students to be included in the analyses (descriptions 
of measures are reported later). For the first step, student 
inclusion criteria were (a) a score at or below the 36th per¬ 
centile on the Dynamic Indicators of Basic Early Literacy 
Skills (D1BELS; Good & Kaminski, 2002) Letter Naming 
Fluency (LNF) measure (i.e., fewer than 6 letters correctly 
named in 1 min) and (b) a score at or below the 37th percen¬ 
tile on the Comprehensive Test of Phonological Processing 
(CTOPP; Wagner, Torgesen, & Rashotte, 1999) Sound 
Matching (SM) subtest. For students who qualified in the 
first step, a second set of criteria were applied that included 
performance (a) at or below the 16th percentile on the Rapid 
Object Naming (RON) measure from the CTOPP or (b) at 
or below the 16th percentile on the Letter Identification 
from the Woodcock Reading Mastery Test-Revised/ 
Normative Update (WRMT-R/NU; Woodcock, 1987/1998). 
These criteria were based on an earlier study (Simmons 
et al., 2011) that accurately identified students most at risk 
of reading difficulty. Specifically, we found that students 
who performed below the 16th percentile on RON or letter 
identification in this second-stage screening were more 
likely to be true positives and to need Tier 2 intervention 
compared to students who performed above those percen¬ 
tile levels. 

Student demographics for ER1-A and ERI-matched 
peers, including baseline scores, are presented in Table 1. 
As illustrated, of the full sample of children who partici¬ 
pated in intervention, 45% were girls and 55% were boys. 
The mean age of participants at pretest was 5.47 years. The 


sample was ethnically diverse, and 18.6% of students were 
identified by teachers as being English language learners. 
Approximately 13.5% of students were eligible to receive 
special education services. 

Intervention Procedures 

In the following, we summarize the ERI (see Simmons 
et al., 2011, for fuller description), the ER1-A (for more 
detail see Coyne et al., in press), and the standard compo¬ 
nents across conditions. 

Early Reading Intervention. ERI (Pearson/Scott Foresman, 
2004) is a supplemental small-group intervention program 
that explicitly teaches phonologic, alphabetic, decoding, 
spelling, and sentence-reading skills to kindergarten stu¬ 
dents at early reading risk. The program includes 126 les¬ 
sons structured in four units that progress from early 
phonemic and alphabetic skills to more complex regular 
and irregular word reading, spelling, and multiple sentence¬ 
reading skills. A typical 30-min lesson consists of seven 
activities, each designed to last 3 to 5 min and to actively 
engage students. Lessons provide explicit scripting for 
introducing, reviewing, and providing corrective feedback. 
New content is taught for 3 days and systematically 
reviewed. Students practice the new skill with the teacher 
and then apply it to discrimination or generalization tasks. 
The program includes four assessments, one at the end of 
each unit, and an instructional pacing chart. 

Students in the ERI condition were initially grouped 
according to similar entry-level skills based on pretest 
assessment data. Starting at Lesson 1, interventionists pro¬ 
gressed through the program sequentially. Students gener¬ 
ally stayed with their initial groups throughout the school 
year; however, in a very few cases, groups were adjusted 
slightly at midyear. Approximately every 8 weeks, mem¬ 
bers of the research team administered ERI end-of-unit 
assessments to evaluate students’ mastery of taught skills. 
End-of-unit assessment results were given to intervention¬ 
ists; however, unlike in the experimental condition, research 
team members did not make instructional recommendations 
based on student performance. 

ERI Adjusted Condition (ERI-A). ERI-A used the same instruc¬ 
tional lessons and materials as the ERI condition, but CEMs 
were added at the midpoint of each unit (approximately 
every 4 weeks) to evaluate students’ mastery of skills taught 
during that period. Research team members met with inter¬ 
ventionists to make instructional adjustments based on the 
student performance data. 

Assessment data were used to determine whether to repeat 
or accelerate lessons as well as to determine which skills to 
review. Our decision rules for adjusting curriculum progres¬ 
sion were based generally on Engelmann’s (1996) theory of 
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Table 1 . Student Demographic and Baseline Scores by Condition. 





ERI-A (n = 136) 


ERI-Matched Peers (n = 

101) 

Variable 

n 

% 

n 

% 

Gender 

Male 

62 

46.6 

45 

44.6 

Female 

74 

54.4 

56 

55.4 

Ethnicity 

American Indian or Alaska Native 

1 

0.7 

0 

0.0 

Black or African American 

40 

29.4 

21 

20.8 

Hispanic or Latino 

50 

36.8 

40 

39.6 

White 

40 

29.4 

38 

37.6 

Other 

5 

3.7 

2 

2.0 

Identified for special education 

14 

10.3 

18 

17.8 

English language learner 

22 

16.1 

22 

21.8 


M 

SD 

M 

SD 

Age 

5.44 

0.31 

5.50 

0.36 

Letter ID 

80.65 

8.19 

79.60 

8.83 

Phoneme segmentation fluency 1 

1.82 

6.02 

2.04 

7.21 

Rapid object naming b 

6.08 

2.19 

6.26 

2.74 

Letter naming fluency 1 

1.20 

1.77 

1.79 

1.94 

Interventionist’s years of experience 1 

1 1.85 

7.41 

1 1.44 

9.70 


Note. ERI = Early Reading Intervention; ERI-A = ERI with ongoing adjustments. 

“Raw score. Standard score. c Ten teachers, who taught in both ERI-A and ERI conditions, were excluded for this mean difference test. 


mastery and acceleration and Bloom’s (1971) theory of mas¬ 
tery learning. Engelmann’s theory specifies that students who 
master material should demonstrate 90% accuracy on assess¬ 
ments of previously taught material. In studies that subscribe 
to Bloom’s theory of mastery learning, the criterion is 80% 
(e.g., Guskey, 2009). To our knowledge, research has not 
definitely established a criterion for mastery, although there 
are generally accepted criteria (e.g., 80%). Our criterion for 
acceleration (90% on two consecutive mastery assessments) 
was rigorous as we wanted to ensure that students were 
highly proficient before allowing them to skip lessons. 
Generally, acceleration schedules involved students complet¬ 
ing two of every three lessons until they reached the final 
unit, in which they were taught each lesson. 

Students with performance ranges of 70% to 89% con¬ 
tinued with the typical lesson progression, which included 
targeted review of letter-sounds already incorporated into 
lessons. Targeted review was designed to enable students to 
progress through the intervention. Performance below 69% 
triggered instructional adjustments including reteaching 
targeted lessons (i.e., based on incorrect letter-sound 
knowledge) and adding a modified letter-sound review 
activity to lessons. After repeated targeted lessons, students 
resumed typical lesson progression until the next mastery 


assessment. Because lesson repetition could involve a sig¬ 
nificant reduction in content coverage, we restricted this 
modification to students who scored less than 70% on con¬ 
tent that had been previously taught. 

When performance did not fall within the performance 
band of the other group members (<70%, 70%-89%, 
>90%), students were regrouped. Because of the small 
number of students at each school, instructional regrouping 
was limited. At the first regrouping occasion, 33.1% of the 
students were regrouped. Following the initial regrouping, 
the average number of students who were regrouped became 
stable and ranged from 7.4% to 13.2% (M= 11.5%) for the 
four subsequent occasions. 

Standard intervention components. To increase comparabil¬ 
ity between ERI and ERI-A conditions, several instructional 
components were standardized. Specifically, (a) both condi¬ 
tions implemented the ERI program (previously described) 
as a supplemental reading intervention, (b) groups in both 
conditions were composed of three to five students, and 
(c) interventionists in both conditions were asked to meet 
with their groups for 30 min, 5 days per week, over the 
course of the intervention period for an equivalent total 
number of sessions. 
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Measures and Assessment Procedures 

Screening and pretests. Approximately 6 weeks into the 
school years, students were screened using the measures 
previously described (i.e., RON, Letter ID, LNF, and SM). 
Students who met the screening criteria were also adminis¬ 
tered the Peabody Picture Vocabulary’ Test-Ill (PPVT-IIl; 
Dunn & Dunn, 1997). 

Rapid automatized naming was measured by the RON 
subtest from the CTOPP (Wagner et al., 1999). In RON, 
students are required to name five objects repeated multiple 
times in a random pattern on two stimulus pages. The score 
is the total time it takes the student to name all stimulus 
objects on both pages. Alternate-form reliability coeffi¬ 
cients as reported by the CTOPP manual range from .79 to 
.82 for students 5 to 7 years old. 

Receptive vocabulary was measured using the PPVT-IIL 
This test requires the student to choose from four examples 
the picture that represents a word presented orally by the 
examiner. Cronbach’s alpha reliability coefficient is .95 for 
5- to 6-year-old students, and retest reliability estimates 
range from .92 to .93. 

Alphabetic knowledge was measured using the Letter ID 
subtest from the WRMT-R/NU and LNF from D1BELS 
(Good & Kaminski, 2002). Letter ID requires the student to 
orally provide the names of upper- and lowercase letters in 
various fonts presented on a stimulus sheet. Split-half reli¬ 
ability in first grade is .94; it is not reported for kindergarten 
students. LNF, in turn, measures the student’s ability to flu¬ 
ently name letters presented on a stimulus page. Students 
are scored on how many letters they can correctly name in 
1 min. Kindergarten alternate-form reliability is .88. 

Curriculum-embedded mastery assessments. ERI mastery 
assessments are criterion-referenced measures that evaluate 
students’ mastery of skills taught in each part of the pro¬ 
gram (i.e., alphabetic, phonemic, word reading, spelling). 
Whereas students in the ERI-A condition were assessed at 
the midpoint and the end of each of the four units of the 
program (approximately every 3—4 weeks), students in the 
ERI condition were assessed at the end of each unit (approx¬ 
imately every 7-8 weeks, depending on the number of les¬ 
sons in the unit). Midunit and end-of-unit measures sampled 
skills introduced in each respective part. Both assessed 
three to four major skills. For example, the mastery assess¬ 
ment at the end of Unit 1 included three subtests that 
assessed phonemic and alphabetic tasks. Letter Names and 
Sounds require the student to provide the correct letter 
name and corresponding sound for each of 11 letters. First 
and Last Sounds in Words requires the student to produce 
the beginning and ending sounds in words presented both 
orally and with an accompanying picture. Finally, for the 
Letter-Sound subtest, the child is given tiles with the letters 
d, f l, m, p, r, s, or t printed on them. The examiner presents 


a page with six pictures and three empty boxes below the 
pictures. After the examiner points to a picture and reads it 
aloud, the student is asked to put the letter tiles for the first 
and last sounds into their corresponding (i.e., first and third) 
boxes. Scores derived from these measures included the 
percentage of total points earned and the percentage of 
points for each skill. Results of end-of-unit mastery assess¬ 
ments were shared with interventionists in both conditions. 
In addition, both the midunit and end-of-unit performance 
of students in the ERI-A condition were shared with inter¬ 
ventionists and subsequently used to inform instructional 
modifications. 

Posttests. The following normative and criterion-referenced 
assessments were administered in April and May of kinder¬ 
garten within 2 weeks of intervention completion. Students 
were given measures of phonological awareness, decoding, 
letter knowledge, word reading, reading fluency, and 
spelling. 

Phonological awareness was assessed using the Blending 
Words (BW) and SM subtests from the CTOPP and Phoneme 
Segmentation Fluency (PSF) from D1BELS. The BW sub¬ 
test requires a student to blend orally presented phonemes 
into the correct word. Cronbach’s alpha is .88 for 5-year-old 
students and .89 for 6-year-old students. In SM, the student 
is provided with a target word and asked to indicate one of 
three words that begins with the same sound as the target 
word. The alpha coefficients for SM are .93 for both 5- and 
6-year-old students. The DIBELS PSF measures the fluency 
with which a student can segment words comprising three or 
four phonemes. Students are scored on the number of pho¬ 
nemes they can correctly provide in 1 min. Alternate-form 
reliability is .88 in kindergarten. 

Letter knowledge was assessed using the Supplementary 
Letter Checklist (SLC) subtest from the WRMT-R/NU. The 
SLC assesses the ability to provide both the letter names 
and sounds of letters presented on a stimulus sheet. No tech¬ 
nical adequacy information is provided in the examiner’s 
manual. 

Decoding was assessed using the Nonsense Word 
Fluency (NWF) subtest from DIBELS and the Word Attack 
(WA) subtest from the WRMT-R/NU. On NWF, students 
are asked to decode nonsense words presented on a stimulus 
page. Students are scored on the number of correct indi¬ 
vidual or word-level sounds they can provide in 1 min. 
Kindergarten alternate-form reliability is .88. On WA, pseu¬ 
dowords are presented on a stimulus page, and the student 
is required to provide the correct pronunciation for the 
word. The median split-half reliability is .87. 

Word reading was assessed using the Word Identification 
(WI) subtest from the WRMT-R/NU. Students are asked to 
correctly read as many words as possible of increasing dif¬ 
ficulty. The median split-half reliability is .97. 
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Reading fluency was assessed using a passage titled 
“Mac Gets Well” (Makar, 1995). Students are scored on 
how many words they can read correctly in 1 min on the 
highly decodable passage. The measure was initially 
designed for early first-grade administration; however, 
Vadasy, Sanders, and Peyton (2006) reported a Cronbach’s 
reliability of .93 for kindergarten students. 

Spelling was assessed using the Test of Written Spelling-4 
(Larsen, Hammill, & Moats, 2005). Students are required to 
spell correctly 10 words presented orally. For 6-year-old 
students, alternate-form reliability is .86 and Cronbach’s 
alpha is .87. 

Assessment Procedures 

All assessments were administered by trained data collec¬ 
tors, including graduate students and members of the 
research team. Training consisted of two 4-hr sessions that 
involved a review of general assessment procedures, mod¬ 
eling of the specific test protocols, paired practice, and 
supervised independent practice of each test. Each data col¬ 
lector met the criterion of 90% accuracy in recording scores 
for a modeled administration of each measure. After data 
collection, two trained individuals scored each testing pro¬ 
tocol independently. 

Data-Analysis Procedures 

We used a three-stage approach to analyze the data: the 
growth-mixture modeling stage, the propensity score 
matching stage, and the multilevel modeling stage. We first 
identified profiles of curriculum-progression adjustment 
and the subgroups of ER1-A students who fit each profile 
using growth-mixture modeling. We then identified matched 
groups of ERI students for each ERI-A subgroup using pro¬ 
pensity score matching procedures. This enabled us to com¬ 
pare differences across end-of-kindergarten reading 
outcomes between each ERI-A subgroup and the corre¬ 
sponding matched ERI group using multilevel modeling. 
The details of these three stages are described below. 

Growth-mixture modeling. We used growth-mixture model¬ 
ing (GMM; B. Muthen, 2004) to identify profiles of curric¬ 
ulum-progression and the unobserved subgroups/latent 
classes of students in each profile. GMM is an exploratory 
approach that can flexibly estimate the potential of unob¬ 
served subpopulations of trajectories with class-specific 
parameter estimates, including the mean/overall average 
trajectory and interindividual and intraindividual differ¬ 
ences in change for each of the unobserved subgroups/latent 
classes (Ram & Grimm, 2009). 

In this study, GMM was used to identify the latent classes 
of curriculum-progression profiles for the ERI-A students, 
which were determined by the total number of lessons they 



Figure I. Latent growth model of the curriculum progression 
for Early Reading Intervention with ongoing adjustments 
students. 

accelerated (skipped) or repeated at the end of each curricu¬ 
lum part, with a total of six parts over the course of the 
study. The proposed latent growth model for the GMM 
analysis is presented in Figure 1. Unlike the simple linear 
growth model with two latent growth factors (i.e., intercept 
and slope factors) and fixed factor loadings, we estimated 
most of the factor loadings of the slope factor, which allows 
for estimation of a nonlinear growth model (Kwok, Luo, & 
West, 2010; Meredith & Tisak, 1990). Given the multilevel 
structure of the current data (i.e., repeated measures on cur¬ 
riculum progression nested within students and students 
nested within intervention groups), we adopted the “Type = 
Complex Mixture” routine in Mplus 6.11 (L. K. Muthen & 
Muthen, 1998-2011), which can adequately account for 
potential dependency by adjusting the standard errors of the 
parameter estimates. To compare different class solutions 
(Nylund, Asparouhov, & Muthen, 2007), the final number 
of latent classes was determined by using the generally rec¬ 
ommended information criteria, namely, Bayesian informa¬ 
tion criterion (BIC) and Akaike information criterion (AIC). 

Propensity score matching. After confirming the final number 
of latent class profiles of curriculum progression for the 
ERI-A students, we identified matched groups of ERI stu¬ 
dents for each of the ERI-A subgroups to be able to test the 
potential effect of adjusting the curriculum progression on 
end-of-kindergarten reading outcomes. Because data were 
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Table 2. Information Criteria for Various Class Solutions. 



One-Class Model 

Two-Class Model 

Three-Class Model 

Four-Class Model 

AIC 

4032.03 

3864.62 

3864.90 

3775.15 

BIC 

4075.72 

3928.69 

3949.37 

3880.00 

ABIC 

4028.27 

3859.10 

3857.63 

3766.12 


Note. ABIC = sample-size-adjusted Bayesian information criterion; AIC = Akaike information criterion; BIC = Bayesian information criterion. 


drawn from a series of studies that compared various inter¬ 
ventions across several years, it was not possible to ran¬ 
domly assign students to each condition (i.e., not all the 
students in our data were randomly assigned to either ERI-A 
and ER1; in particular, for the first cohort, only ERI was 
available). The data set containing 136 ERI-A students was 
separated into four different classes/subgroups based on the 
growth-mixture analysis. Each of the four ERI-A subgroups 
was then separately combined with a data set of 101 ERI 
condition students to estimate the propensity scores. Propen¬ 
sity scores, defined as a student’s probability of being in the 
corresponding ERI-A condition, were estimated based on 
the 14 covariates measured at the baseline, including both 
demographic variables (i.e., gender, age, limited English 
proficiency, ethnicity, special education status, and sites) 
and the baseline achievement scores (i.e., blending words, 
sound matching, rapid object naming, letter naming fluency, 
PPVT, letter identification, word identification, and word 
attack). By mimicking a randomized experimental design, 
which controls for any potential preexisting differences on 
these variables between the four different groups of ERI-A 
and ERI children, propensity matched samples enable us to 
estimate the true difference between ERI-A and ERI condi¬ 
tions on the target reading outcome measures. 

The propensity scores were computed utilizing logistic 
regression analysis by using the above 14 covariates as pre¬ 
dictors (see Note 1). Based on the propensity scores, the 
ERI-A and ERI students were then optimally matched 
within each of the four ERI-A conditions. We used the one 
to one matching scheme (Ming & Rosenbaum, 2001) with 
the PROC ASSIGN routine in SAS (v. 9.3) to create the 
optimally matched groups. We also used caliper distance of 
0.25, indicating that any pair of matched groups did not dif¬ 
fer by 0.25 standard deviations in their propensity scores. 
We then checked if the matching result provided good bal¬ 
ance between the ERI-A and ERI groups. In other words, 
for two groups to be balanced, they should not differ on the 
14 variables because they were matched based on having a 
similar probability of being assigned. We found that the 
four matched data sets had good balance. 

Multilevel modeling. We adopted multilevel modeling to test 
the differences between each ERI-A subgroup and the cor¬ 
responding matched ERI group on end-of-kindergarten 
reading outcomes. Because nonindependent observations 


were used because of the nesting structure in our data (i.e., 
students nested within different intervention groups), multi¬ 
level modeling (Hox, 2002) was applied to analyze the data, 
as it takes nonindependence into account. All multilevel 
models were analyzed using the MIXED routine in IBM 
SPSS V20.0. Restricted maximum likelihood was used to 
estimate all the models, and two-tailed tests were used to 
evaluate intervention effects. The corresponding equations 
of the multilevel model are presented in Appendix A. 

In addition to the tests of significance (i.e., t test) of the 
difference between the ERI and the ERI-A conditions on the 
posttest measures, we used Hedges’s (2007) proposed effect 
size (5 ) to evaluate the practical significance of differences 
in cluster-randomized designs. The equation of 8 is pre¬ 
sented in Appendix B. To control for the potential inflated 
Type I error rate because of the multiple comparisons, we 
used the Benjamini-Hochberg (BH) correction (Benjamini 
& Hochberg, 1995), as recommended by the What Works 
Clearinghouse (WWC, 2008). In accordance with the 
approach recommended by the WWC, we report compari¬ 
sons that are statistically significant and also interpret treat¬ 
ment effects that are greater than .25 but not statistically 
significant as “substantively important” (p. 23). 

Results 

Identifying the Number of Latent Class Profiles 
of Student Performance!Curriculum Progression 

To identify the optimal number of latent class profiles link¬ 
ing student performance and curriculum progression, we 
estimated one-class, two-class, three-class, four-class, and 
five-class models separately. We did not obtain the con¬ 
verged solution from the five-class model, which implied 
that the four-class solution was sufficient to account for het¬ 
erogeneity in the curriculum progression trajectories 
(Enders & Tofighi, 2008; Nylund et al., 2007). The corre¬ 
sponding AIC and BIC values for different class solutions 
are presented in Table 2. By comparing it with other class 
solutions, the four-class solution resulted in the lowest val¬ 
ues for both AIC and BIC, which also indicated that the 
four-class model outperformed other models and fit our 
data the best. 

Figure 2 presents the four latent profiles based on the 
four-class solution across the six measurement points. 
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—a— Decelerated repetitions to 

standard progression (n = 81) 
—Early/sustained acceleration 
(n = 24) 

- Minimal lesson repetition 

with targeted review (n = 24) 

-Standard progression with 

targeted review (n = 7) 


Figure 2. Four latent classes from growth-mixture modeling 
using the number of accelerations across six time points. 


Among the total 136 students in the ERI-A condition, a 
majority (n = 81) belonged to a class we labeled the “decel¬ 
erated repetitions to standard progression” group. In this 
class, students repeated four lessons at the first assessment 
point. The pattern of repetitions steadily decreased, indicat¬ 
ing improved performance on CEMs. At the fourth assess¬ 
ment point, students in this class were on track for typical 
lesson progression. A total of 24 students were in the “early/ 
sustained acceleration” group. The performance/progression 
profile of this class indicated a few repetitions at the first 
assessment point, after which students consistently per¬ 
formed at 90% or higher on CEMs and accelerated as many 
as seven lessons per unit. Another 24 students belonged to the 
“minimal lesson repetition with targeted review” group, 
which repeated, on average, one lesson every measurement 
point. Finally, 7 students were in the “standard progression 
with targeted review” group, indicating they consistently 
scored between 70% and 89% on mastery measures and 
received the standard implementation schedule. Table 3 
shows the corresponding parameter estimates (i.e., the means 
of the latent growth factors and the estimated factor loadings) 
of each of the four classes, as presented in Figure 2; Table 4 
shows the means and standard deviations of the poststudy 
reading outcomes for all four classes respectively. 

Matching Peers in the ERI Condition Using 
Propensity Score Analysis 

We used the propensity score matching procedure 
(Rosenbaum & Rubin, 1983) to select the comparison units 
(i.e., ERI students) that were similar in terms of the sample 
characteristics. To produce the propensity score, we used 14 
covariates, including demographic variables (i.e., gender, 
age, English language learner status, special education sta¬ 
tus, ethnicity, and site) and baseline reading skills (i.e., 
blending words, sound matching, rapid object naming, let¬ 
ter naming fluency, vocabulary, letter identification, word 
identification, and word attack). Based on the propensity 
score, 75 ERI students were matched to the group of “decel¬ 
erated repetitions to standard implementation” (n = 81 


students), whereas 22 ERI students were matched to the 
“early and sustained acceleration” group containing 24 
ERI-A students. Similarly, 15 ERI students were matched to 
the “minimal lesson repetition with targeted review” group 
(n = 24 ERI-A students), whereas two students in ERI were 
matched to the “standard progression with targeted review” 
group, which consisted of only 7 ERI-A students. 

We conducted the sensitivity analyses (Robins, 
Rotnitzky, & Scharfstein, 1999) using t tests and chi-square 
tests to check the quality of the matched groups by examin¬ 
ing the possible initial group difference between the 
matched ERI and the corresponding ERI-A conditions on 
all the demographic variables, as well as the baseline read¬ 
ing skills. No statistically significant difference was found 
on any of these variables between the two conditions across 
all four classes. Hence, the matching procedure was suc¬ 
cessfully employed. As a result, if any significant difference 
on end-of-kindergarten outcomes was found, we could attri¬ 
bute it to being the result of the curriculum progression 
adjustments. 

Testing Difference Between ERI-A and ERI on 
End-of-Kindergarten Reading Outcomes 

Table 5 presents the results of the multilevel analyses for 
testing the effect of performance/progression adjustments 
by the four latent class profiles separately, and the standard¬ 
ized effect size, 5 , for each comparison and outcome mea¬ 
sure. As illustrated, within the comparisons between the 
ERI-A “decelerated repetitions to standard progression” 
group and the corresponding matched ERI group (n = 156), 
the ERI-A group scored significantly higher than the com¬ 
parison group on the following posttest measures after 
adjusting for the covariates and controlling for the compar¬ 
ison-wise Type 1 error rate using the BH correction: 
WRMT-R/NU SLC-Sound, PSF, Word ID, and Oral 
Reading Fluency. Effect sizes for these measures were also 
substantively important (effect sizes greater than .25) along 
with the effect size for WRMT-R/NU SLC-Name and 
CTOPP BW. 

Within the comparisons between the ERI-A “early/sus¬ 
tained accelerations” group and the corresponding matched 
ERI group, the ERI-A scored significantly higher on five 
outcome measures (i.e., WRMT-R/NU SLC Letter and 
Sound, CTOPP SM, CTOPP BW, and Word ID) after con¬ 
trolling for the covariates and Type I error rates by BH cor¬ 
rection. Results showed that the ERI-A students in general 
outperformed their matched ERI counterparts with substan¬ 
tively important effects on all measures (5 range from 0.36 
to 1.25). 

For the “minimal repetition with targeted review” group 
(n = 35) who consistently performed between 70% and 89% 
on mastery assessments, no statistically significant between- 
group differences were found. Effect sizes varied greatly by 
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Table 3. Parameter Estimates for Growth-Mixture Analysis With Four-Class Solution. 


Decelerated Repetitions to Early/Sustained Minimal Lesson Repetition With Standard Progression With 



Standard Progression (n = 81) 

Accelerations (n 

= 24) 

Targeted Review (n 

= 24) 

Targeted Review (n = 7) 

Estimate 

SE 

p Value 

Estimate 

SE 

p Value 

Estimate 

SE 

p Value 

Estimate 

SE 

p Value 

“ll 

4.102 

0.600 

.000 

2.553 

0.592 

.000 

0.756 

0.418 

.071 

0.932 

0.816 

.254 

Si 

-1.309 

0.541 

.016 

-1.922 

0.851 

.024 

-0.294 

0.322 

.362 

-0.169 

0.225 

.455 

^32 

2.387 

0.984 

.015 

4.067 

1.858 

.029 

2.550 

2.394 

.287 

-9.961 

5.816 

.087 


2.884 

1.223 

.018 

4.81 1 

1.972 

.015 

13.626 

7.458 

.068 

2.077 

0.857 

.015 

^52 

4.247 

1.852 

.022 

3.701 

1.530 

.016 

4.795 

2.421 

.048 

3.001 

1.410 

.033 


3.720 

1.608 

.021 

2.159 

0.942 

.022 

3.921 

1.772 

.027 

3.286 

1.582 

.038 


outcome measure, with some favoring the ERI-A group 
(spelling § = 0.50, blending words 5 = 0.36) and others 
the ERI condition (word identification 5 = 0.77, oral read- 

W 

ing fluency 6 =1.16). The pattern of findings was similar 
for the “standard progression with targeted review” group, 
with no statistically significant differences. Letter-sound 
knowledge, phonemic segmentation fluency, blending 
words, and spelling effect sizes exceeded .25 and favored 
the ERI-A condition. Nonsense word fluency, word attack, 
and word identification effect sizes were greater in the ERI 
conventional implementation. 

Discussion 

To date, few studies have examined the impact of intensify¬ 
ing interventions by adjusting students’ progression through 
an intervention curriculum in response to their mastery of 
taught content (e.g., Denton et al., 2010). Although adjust¬ 
ing intervention in response to content mastery is recom¬ 
mended in research-to-practice guides and articles (Gersten 
et ah, 2009; Mellard, McKnight, & Jordan, 2010), to our 
knowledge no studies have specifically compared an inter¬ 
vention implemented conventionally with the same inter¬ 
vention implemented with ongoing adjustments based on 
student response. The analyses presented in this study were 
motivated by findings from two randomized trials that pro¬ 
vided encouraging evidence that adjusting curriculum pro¬ 
gression based on student mastery data resulted in improved 
outcomes with effect sizes that ranged from 0.29 to 0.76. 
The primary findings from previous studies, however, did 
not identify which curriculum adjustments were associated 
with enhanced achievement. 

Using GMM, we identified and examined student perfor- 
mance/curriculum progression profiles that emerged for dif¬ 
ferent groups of students when mastery assessment data 
were used to make instructional adjustments. We compared 
subgroups of students from each latent profile to propensity 
matched peer groups to investigate whether the performance 
of students receiving the ERI kindergarten reading interven¬ 
tion with ongoing curriculum adjustments differed from that 
of students receiving the ERI intervention implemented 


conventionally. Findings indicated that disaggregating the 
overall effects of instructional adjustments from prior stud¬ 
ies offers important information for intensifying interven¬ 
tions. Analyses identified four profiles that characterized 
students’ progression through ERI-A. Two of these profiles 
indicated statistically significant advantages over conven¬ 
tional intervention implementation with effects sizes rang¬ 
ing 0.13 to 1.25. The pattern of performance in the two 
smallest profile groups was less clear. 

The Effects of Distinctive Student Performance/ 
Curriculum-Progression Profiles 

Decelerated repetitions to standard implementation. The larg¬ 
est class comprised 81 (60% of ERI-A participants) stu¬ 
dents who scored, on average, below 70% on early mastery 
checks (e.g., 68% on Point 1 and 76% on Point 3) followed 
by improved performance (e.g., 81%) at the fifth adjust¬ 
ment period. Results of multilevel analyses indicated that 
students in this profile outperformed 75 matched peers from 
the ERI condition on 4 of 10 outcomes with effect sizes 
ranging from 0.13 to 0.48. (Students in this class were the 
lowest performing of the four groups at posttest.) Findings 
from this profile suggest that adjusting instruction to 
develop mastery enhances learning outcomes over conven¬ 
tional progression through ERI; however, we consider the 
differences of moderate magnitude. On average, students in 
this group repeated four lessons at the first adjustment 
period with steady decreases in the number of lesson repeti¬ 
tions. We hypothesize that the initial repetitions were nec¬ 
essary for students to develop building-block skills (i.e., 
new letter-sound correspondences), which they then 
applied to more complex skills such as word identification 
(8 = 0.45) and oral reading fluency (8 = 0.48). 

Early and sustained acceleration. A second class was the 
group of 24 students who received ERI-A with early and 
sustained lesson acceleration. Consistently, the average per¬ 
formance of this group was greater than 90% on all assess¬ 
ments, enabling interventionists to compress the curriculum 
(Reis, McCoach, Little, Muller, & Kaniskan, 2011) and skip 




___^ 


on' 

^^ 

— 

— 

p 

O 

p 

1^ 

On' 

O 

— 








0 

0 

0 

O 

0 

P 

p 

O 

rv 

NO 

NO 

On' 

fN 

fN 

fN 




ro 


o' 

oo' 

p 

S' 

iv. 

p 

ro 

O 

— 

p. 

fN 

fN 





ON 

0 

NO 

O 

(N 

0 

CO 

O 

vb 

00 


oC 

fN 

rs 

fN 




o? 



ro 

LO 

o' 

CO 

rN 

l< 

p 

LO 

on' 


— 




-- 

N - 

NO 

ON 

ro 

NO 

p 

0 

rv 

H 

—- 

On' 

CO 

(N 

rN 

ro 




fvT 


On' 


p 

LO 

ON 

p 

NO 

p 

—- 

ro 


rN 





ON 

— 

fN 

— 

(N 

1^ 

N- 

fN 

rv 

rv 

00 

oC 

fN 

fN 

N" 






Co' 

Lo' 

1^ 


LO 

fN 

00 

IV. 

|v! 

p 


— 




'—' 

ON 

LO 

LO 

ON 

p 

p 

p 

O 

rN 

—I 

NO 

On' 

fN 

rN 

ro 






LO 

o' 

NO 


p 

p 

hs! 

p 

H 

ro 


fN 

N-■ 

" 

v —•" 

'—' 

00 

N" 

0 

ro 

00 

LO 

p 

ro 

NO 

00 


O 

ON 

rN 

N" 




LO 


— 

oo' 

00 

ro 

NO 

ro 

On' 

p 

NO 

0 


— 




■—' 

00 

IV. 

N- 

ro 

p 

(VJ 

p 

00 

fN 

—I 

fN 

CO 

ON 

rN 

ro 




oT 


oo' 


NO 

On' 

CO 

p 

_' 

p 

p^ 

NO 

fN 

fN 





1^ 

0 

LO 

On 

rNj 

p 

—■ 

ON 

■*■’ 

N" 

On' 

CO 

fN 

rN 

ro 




V. 

u 

c 


V) 

<A 

Q) 


15 

15 

_3 


u 

0 

l_l_ 


<u 

_c 

u 

<D 

_c 

u 

c 

0 

"■M 


a> 

"0 



E 

C3 

z 

c 

3 

0 

to 

c 

<1> 

E 

-O 

bO 

_c 

s_ 

0 . 

tXO b 

b£ 

0) 

1 c 


LO LO 

O p — 

t ro CO 

fN O O 


•£3 c V « 

ctf —' a3 3 <■ 

u z S 5 = ^ 

•3 ^ ^ ^ w 

ci S- •— 0 

V \- C -o (J 

T3 o n3 

— 2_ txo 0 a; u 

■p .E « *- ^ 

o ^ "3j g ? 

^ £■ o 


8 § 2 

g -5 8 

£ I -g 

b ^ c 
cs c b 
V) OJ w 

§ £i° 

— cu 

5 £ £ 

— — o 

< ¥ a 

a ■g 5 

z & 


265 



Decelerated Repetitions to Early/Sustained Minimal Lesson Repetition With Standard Progression With 

Standard Progression (n = 156) a Accelerations (n = 46) b Targeted Review (n = 35) c Targeted Review (n = 9) d,e 
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targeted lessons that reviewed and applied skills. Students 
in this group significantly outperformed their matched peers 
on five outcomes, and effect sizes ranged from 0.36 to 1.25 
on the full complement of outcome measures. 

A premise of acceleration is that by eliminating skills or 
lessons that do not need to be taught, more advanced con¬ 
tent can be incorporated enabling students to acquire more 
complex skills. Because we worked with a fixed set of 126 
lessons, we may inadvertently have constrained accelera¬ 
tion effects. The deceleration in the curriculum progression 
graph (see Figure 2) indicates that as students approached 
the end of the program, lesson acceleration decreased, an 
artifact of both the intervention (e.g., there were no more 
lessons to accelerate through) and the desire to control for 
total number of lessons that students received across condi¬ 
tions. Based on these findings, further research on the ben¬ 
efits of lesson acceleration for students in Tier 2 intervention 
is clearly warranted. 

Effects of minimal repetition and targeted lesson review. Two 
profiles, 3 and 4, closely resembled standard implementa¬ 
tion. The third profile reflected a curriculum progression 
that hovered slightly below zero, indicating a progression 
that averaged less than one repetition per curriculum unit 
with targeted skill review. Group performance averaged 
between 82% and 87% at all assessment points. The fourth 
trajectory included only seven students, and, on average, 
performance on part tests improved over time from 68% at 
the first point to 81% at the fifth checkpoint. Curriculum 
progression included repetitions, standard progression, and 
accelerations. 

Findings from both trajectories indicated no statistically 
significant differences between the performance of adjusted 
groups and those of peers who received the conventional 
intervention. Effect sizes, however, revealed a range of per¬ 
formance differences that varied between ERI-A and ERI 
groups. These were the smallest profile groups, and further 
research is warranted to examine performance variability. 

Limitations of the Study 

We want to acknowledge several limitations of this study. 
First, the study was based on studies of only one supple¬ 
mentation intervention (i.e., ERI) in one grade, kindergar¬ 
ten. To understand whether these types of mastery-based 
curriculum-progression adjustment profiles replicate and 
influence reading achievement, research is needed with dif¬ 
ferent interventions across a range of grades. A second limi¬ 
tation is the method we used to operationalize RTI, leaving 
unanswered the following questions: Were the procedures 
and decision rules used to accelerate students too stringent? 
Was the decision to provide targeted review of letter-sound 
correspondences as opposed to other skills too restricted or 
the interval too broad? 


Our cut points of 90% for acceleration and below 70% 
for lesson repetition, although instructionally defensible, 
also require further study. Moreover, the decision to provide 
targeted review and no lesson repetition for students per¬ 
forming in the range of 70% to 89% needs further examina¬ 
tion. This is a broad range of performance, and our targeted 
review may need to be intensified or alternate methods of 
intensifying intervention explored. To our knowledge, how¬ 
ever, there are no studies that provide definitive decision 
rules on how to adjust intervention in response to student 
performance. 

A final limitation relates to balancing science and the 
realities of intervention. Our intent in the ERI-A condition 
was to group students with similar skill profiles and adjust 
lesson progression accordingly. Our efforts were, at best, 
reasonable approximations. In some schools, there were too 
few groups to appropriately place students with similar 
skills and needs. In other cases, when a student’s perfor¬ 
mance fell between two groups, we may have erred by plac¬ 
ing a student in a group that was too high or too low, thus 
influencing his or her performance. 

Conclusion and Implications 

There is emerging but incomplete scientific evidence for 
how to intensify Tier 2 intervention for students at risk of 
reading difficulties (Gersten et al., 2009). Findings from 
the present study indicate that two types of progression 
adjustments based on student response to mastery assess¬ 
ments differentially improved some but not all reading and 
reading-related outcomes. The strongest benefits of cur¬ 
ricular adjustments, based on statistical and practical 
effects, were for students who scored at or above 90% cor¬ 
rect on mastery assessments and who were accelerated 
through an early reading intervention by skipping targeted 
lessons. Although their matched peers in the unadjusted 
condition also achieved positive outcomes, findings sug¬ 
gest that we can enhance intervention efficacy by using 
data to expedite lesson progression. Thus, we found a 
moderate benefit of curriculum adjustments for students 
who scored below 70% correct on early mastery check¬ 
points and repeated key lessons compared to matched 
peers in the unadjusted condition. Findings indicated a 
steady improvement on mastery assessments from 68% to 
81% for students who repeated early lessons (covering 
foundational skills) with decelerating lesson repetition 
necessary as the program progressed. Performance pro¬ 
files that most closely resembled standard implementation 
yielded non-statistically significant differences but mixed 
effect sizes. 

Although significant strides have been made in address¬ 
ing the needs of children who are likely to experience read¬ 
ing difficulty, findings from this study suggest the need for 
further research that addresses the intervention, grade, and 



268 


Journal of Learning Disabilities 48(3) 


RTI limitations identified. Specifically, we are in need of 
answers to such questions as these: Will the effects of accel¬ 
eration and lesson repetition based on student performance 
replicate in different intervention programs that use differ¬ 
ent forms of assessment? What are the most effective meth¬ 
ods for improving the performance of students who achieve 
mastery of 70% to 89% of intervention content? 

In conclusion, we are reminded of a quote from a coordi¬ 
nator with whom we worked in a statewide effort to improve 
reading instruction and achievement. Her advice seems par¬ 
ticularly appropriate in this context: “You want to teach 
children as quickly as you can but as slowly as you must.” 
Although research on intensifying intervention through per¬ 
formance/progression is relatively new, findings suggest 
the promising results of responsive intervention, that is, 
intervention that systematically uses student data to adjust 
instruction and enhance performance. 

Appendix A 

We compared the performance of the Early Reading 
Intervention (ERI) with ongoing adjustments (ERI-A) stu¬ 
dents in each latent class to that of the corresponding pro¬ 
pensity matched ERI students from the ER1 condition using 
multilevel analysis (Hox, 2002) with the following set of 
models, 

Level 1 (Student-Level) Model 

Posttest = B +6 Pretest + B PPVT + 

ij K 0j K lj lj K 2 j ij 

P 3 .Age.. + P Gender.. + P 5j Hispanic.. + 

B, African American.. + B„.Special .. .. 

%j - y h 7j f - (Al) 

ed. + P s .Bilingual.. + P 9 .Last Lesson 

Number + B Number of Lessons + 

y noj ij 

B, Propensity Score.. + r. 

i ij J ij ij 

where i represents the i-th student and j represents the j-th 
group. Posttest ; k is the score of one of the posttest measures 
for the i-th student from the j-th group. 

In this student-level model, covariates included the corre¬ 
sponding pretest score (Pretest.. k ), PPVT score (PPVT ) as 
an indicator of the student’s receptive language ability, plus 
demographic variables consisting of student age (Age ), gen¬ 
der (Gender), ethnicity (represented by two dummy-coded 
variables: Hispanic.^ and African American.^), special educa¬ 
tion services (Special_ed.. k ), bilingual status (Bilingual.. k ), and 
two dosage variables, the number of lessons students received 
(Number of Lessons.^) and the last lesson number (Last Les¬ 
son Number ). We also included the propensity score gener¬ 
ated from the logistic regression model as a covariate. The 
within-group random error is r.., and the corresponding vari¬ 
ance, V(r ) = a 2 , captures the within-group variation. 


For those poststudy-only measures (i.e., WRMT-R/NU 
Letter Name and Letter Sound Checklists, Test of Written 
Spelling-4, and oral reading fluency), we used the untimed 
letter ID pretest score as their pretest score given that 
untimed letter ID was highly correlated with all these post¬ 
study reading measures. 

The intervention group-level models were specified as 
shown below: 

Level 2 (Group-Level) Models 

B = y + y ERI-A + y Interventionist’s 

K 0j '00 '01 j 02 

Experience. + u 

and 

^lj V ^2j ^20’ ^3j ^30’ ^4j ^40; 

^5j ^50; ^6j ^60; ^7j ^70; ^8j ^80; (A2) 

^9j ^90;Pl0j ^100; Pllj "^110 

In Equation A2, ERI-A. is a dummy-coded variable with 
1 as ERI-A condition and 0 as ERI condition. The interven¬ 
tionist’s years of teaching experience was also controlled 
at the group level. The between-group random effect is u , 
and the corresponding variance, V(u ) = t qo , captures the 
between-group variation. Intraclass correlations (ICCs; i.e., 
between-group variation over the total variations) were 
calculated. The average ICC of all measures across four 
classes ranged from .24 to .48, a common range in educa¬ 
tional research (Hox, 2002). The nonzero ICCs provided 
support for the dependency of observations in same clusters 
and the need to use multilevel models to adequately handle 
the nonindependency in our data. 

Appendix B 

To evaluate the practical significance of the difference on 
the poststudy reading outcomes between ERI-A and the 
corresponding ERI condition (i.e., ERI), we used Hedges’s 
(2007) proposed effect size (i.e., 8 ), which is specifically 
designed for cluster-randomized studies. It is computed 
using the following equation, 

5 = (Bl) 

where y is the mean difference between ERI-A and ERI 
after adjusting for the demographical and pretest covariates. 
Within-group variance (i.e., a 2 ) was adopted from the 
unconditional multilevel model in which no covariate/pre¬ 
dictor was included. All 237 students were included for 
conducting the unconditional model. 
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Note 
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+ WA + site. 
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